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Abstract: In the current landscape of higher education, the quality of teaching plays a crucial role in supporting the 
comprehensive development of students. To ensure the effectiveness of the learning process, evaluating lecturers based on 
student opinions is an essential means of providing feedback and optimizing the learning experience. This paper focuses 
on constructing a lecturer ranking system, particularly in the context of a specific course through the evaluation process 
from students. Four different methods were employed to assess lecturers, including the PSI method, SRP method, RAM 
method, and PIV method. The evaluation results using these four methods were compared with each other and also with 
the traditional evaluation approach currently utilized in the educational institution. The achieved results demonstrate that the 
approach outlined in this paper is highly suitable for determining the rankings of lecturers when teaching individual courses. 

Keywords: Lecturers ranking, MCDM, PSI, SRP. RAM, PIV 


Introduction 


Ranking lecturers when multiple individuals teach a course plays a crucial role in ensuring the qual- 
ity of education and the professional development of the teaching staff (Ventista and Brown, 2023; Ekinci 
et al., 2022). Each lecturer brings their own perspective and teaching style, and evaluating the quality of 
their teaching can help schools and students make more informed decisions about how they approach 
a specific course. Ranking lecturers not only helps identify the best teachers but also highlights issues 
that need addressing in the teaching process (Munna and Kalam, 2021). Highly rated lecturers may be 
considered for roles such as department heads, sharing effective teaching methods with colleagues, or 
even being invited to teach additional classes to expand their positive impact. Conversely, lecturers facing 
challenges may require support, additional training, or, in some cases, a reevaluation of their teaching 
abilities in that specific course. Ranking lecturers also aids schools in managing lecturer resources more 
effectively (Girvan et al., 2016). If any lecturer is assessed as unsuitable or not meeting the requirements 
for teaching a specific course, decisions to minimize or cease their teaching assignments may be made to 
optimize the quality of education. This not only helps avoid issues with teaching quality but also optimizes 
the professionalism and motivation of the teaching staff (Oliver and Reschly, 2017). In summary, ranking 
lecturers when multiple individuals teach a course is not only an assessment tool but also a means of 
managing and developing lecturer resources, ensuring diversity and quality in the teaching process. 

If lecturer evaluations rely solely on the subjective opinions of managers or focus on specific criteria 
such as the number of published papers or participation in projects, important limitations may arise due 
to subjective managerial opinions and overlooking the complexity and diversity of lecturer roles. Some 
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universities in Vietnam have implemented lecturer evaluations based on students’ evaluations of them. This 
approach is relatively novel, where lecturer evaluations are conducted within the university’s education 
system, requiring each student to evaluate the lecturer who taught them if they wish to know their grades 
for that course. To clarify, students must evaluate their lecturers through a set of criteria provided by the 
management agency to access their grades for a specific course. Once all students have completed grad- 
ing lecturers on each criterion, the education management system calculates the average value for each 
criterion for each lecturer and then computes the overall average score for the lecturer. However, calculating 
the overall average value for all criteria for each lecturer overlooks the importance of individual criteria. This 
means that not all criteria for evaluating lecturers are equally important; their significance varies depending 
on the characteristics of each course. For example, in practical courses, lecturers need good operational 
skills, while in theoretical courses, lecturers need extensive academic knowledge. To objectively evaluate 
lecturers based on multiple criteria, multi-criteria decision-making (MCDM) methods are considered, as they 
have been applied successfully in various fields (Kalyan and Pramanik, 2019; Malik et al., 2021; Ghorui et 
al., 2021). 

MCDM is a technique for ranking available options to facilitate the selection of the best option and 
avoid the worst ones (Truong et al., 2023). With more than 200 existing MCDM methods, they have been 
applied to rank alternatives in various fields, from industry to transportation, healthcare, construction, and 
more (Trung and Tung, 2022). In the field of education, the application of MCDM methods has also been 
found in some studies. The PROMETHEE method has been used to rank teachers in a competition for 
excellent teaching (Monalisa and Kusnawi, 2017). Two methods, AHP and ARAS, have been used to rank 
lecturers based on criteria such as work experience, academic qualifications, etc. (Akmaludin et al., 2023). 
Ranking university departments using the RAPS method has been implemented. This research used criteria 
to describe each department, such as research productivity, total number of published chapters and articles 
in Scopus, total number of professors, total number of associate professors, etc. (Bafail et al., 2022). The 
TOPSIS method was used to select websites for online teaching during the Covid-19 period (Toan et al., 
2021). The TOPSIS method has also been used to evaluate students’ academic performance (Sirigiri et al., 
2015). Applying the VIKOR method to rank universities based on student opinions has been carried out in 
(Ayyildiz et al., 2023), and so on. 

This article employs MCDM methods to evaluate lecturers in each course based on student evalua- 
tions. The four MCDM methods used in this study include the PSI, SRP, RAM, and PIV methods, chosen for 
their distinct characteristics. The PSI method requires data normalization but does not require weighting for 
criteria (Maniya and Bhatt, 2010; Do et al., 2023). Recent studies have applied this method in various fields, 
such as ranking transportation companies (Ulutas et al., 2021), selecting materials in mechanical manu- 
facturing (Dua, 2024), developing decision systems for awarding student scholarships (Arifin and Saputro, 
2022), and choosing plastic injection molding machines (Trung et al., 2024a). In contrast, the SRP method 
does not require data normalization but necessitates weighting for criteria (Zakeri et al., 2023), and is rec- 
ommended for cases where the number of alternatives to be ranked is greater than 5 (Zakeri et al., 2024). 
Recent publications have successfully applied this method in selecting regression models for surface rough- 
ness in grinding (Thinh and Dua, 2024), choosing 3D printing materials (Mian et al., 2024), and evaluating 
the financial health of banks (Trung et al., 2024b). 

RAM is a relatively new method, first introduced in September 2023, known for its ability to balance 
favorable and unfavorable criteria (Sotoudeh-Anvari, 2023). This method has been applied in various fields 
such as selecting cultivation methods for mushrooms (Trung et al., 2024c), choosing materials for gear 
manufacturing, materials for screw manufacturing, and lubricants for two-stroke engines (Dua et al., 2024), 
selecting options in mechanical manufacturing (Trung et al., 2024d), and ranking universities (Do, 2024). 
PIV is known for minimizing the phenomenon of rank reversal (Mufazzal and Muzakkir, 2018). Some of the 
most recent studies have successfully applied this method for selecting materials for crankshaft manufactur- 
ing (Nguyen et al., 2024), choosing materials for connecting rod manufacturing (Thinh and Mai, 2023), and 
comparing the impact of the Covid-19 pandemic on various countries (Komasi et al., 2024). When apply- 
ing the RAM and PIV methods, both tasks of data normalization and weighting for criteria must be carried 
out. The normalization methods for data in RAM and PIV are different, and it’s important to note that the 
data normalization method significantly affects the ranking of alternative options that need to be prioritized 
(Trung, 2022; Ha, 2023). The selection of the four methods, PSI, SRP, RAM, and PIV, takes advantage of 
their different characteristics and outstanding merits, aiming to rank lecturers accurately and objectively. 
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Materials and Methods 


PSI Method 


The application of the PS/ method to rank alternatives is carried out in eight steps as follows 
(Maniya and Bhatt, 2010; Do et al., 2023). 

Step 1: Construct a decision matrix with m rows and n columns, where m and n represent the num- 
ber of alternatives to be ranked and the number of criteria for each alternative, respectively. Let Xi denote 
the value of criterion j for alternative /, with j = 7 ton, /= 1 to m. The letters B and C are used to signify the 
benefit and cost criteria, respectively. 

Step 2: Normalize the data using formulas (1) and (2). 


a eee 
ny = pre if jeB (1) 
nz = 2 ii if jeCc 2 
iy Xy (2) 
Step 3: Calculate the average value of the normalized data using formula (3). 
1 m 
n= 2 ny (3) 
Step 4: Determine the priority value from the average value using formula (4). 
Q; = » [75 — nj? (4) 
Step 5: Identify the deviation in priority values using formula (5). 
9=1- 9; (5) 
Step 6: Determine the overall priority value for criteria using formula (6). 
ae. 
Am ee m 
Step 7: Calculate the priority selection index for each alternative using formula (7). 
a,= 2 ni - F; (7) 


Step 8: Rank the alternatives based on the principle that the best alternative is the one with the highest 0. 


SRP Method 


To rank alternatives using the SRP method, four simple sequential steps need to be applied (Zakeri 
et al., 2023; Zakeri et al., 2024). 

Step 1: Similar to Step 1 of the PS/ method. 

Step 2: Internally rank the alternatives for each criterion. For criterion j, the rank of alternative / is 
denoted as re Note that the internal ranking of alternatives uses only natural numbers, meaning if IN. This 
can be illustrated through a simple example. Suppose there are five alternatives to be ranked, namely Al, 
A2, A3, A4, and AS. There are three criteria describing each alternative, namely C1, C2, and C3. Among 
them, C1 and C2 are two benefit criteria, while C3 is a cost criterion. The illustrative data for this example 
is presented in Table 1. 
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Table 1. Data to illustrate Step 2 of the SRP method 


Alternative C1 C2 C3 
Al 7 3 8 
A2 5 6 2 
A3 7.5 4 4 
A4 5.4 6 2 
Ad 4.2 3 2 


The internal ranking of alternatives is performed as follows. 
* For C1 (B form): Since the values of this criterion for all five alternatives are different, the rankings of 
the alternatives are arranged in decreasing order of the criterion values in the alternatives, i.e., r,, = 1, 
221, =3,0,>4, andr, =. 
For C2 (B form): Since the criterion values at A2 and A4 are both equal to 6 


ly. = 2. As the criterion values at A1 and AS are also equal, r,, = r,, = 3. 


For C3 (C form): Since the criterion values at three alternatives A2, A4, and AS are equal, r,,= 1, = I, 
= 1, followed by r,,= 2 and r,, = 3. 


Fog = Vay = 1, followed by 


Step 3: Calculate the score for each alternative using formula (8), where w, is the weight of criterion j. 


S;= dry; (8) 


j=1 


Step 4: Rank the alternatives in ascending order based on their scores. 


RAM Method 


Ranking alternatives using the RAM method is performed in six steps as follows (Sotoudeh-Anvari, 
2023). 
Step 1: Similar to Step 1 of the PS/ method. 


Step 2: Normalize the data using formula (9). 


ie are (9) 
Step 3: Calculate the normalized values considering the weights of the criteria according to (10). 
be ald Hid (10) 


Step 4: Calculate the sum of normalized scores considering the weights of the criteria using (11) and (12). 


Su=) yey if JEB (11) 
j=1 
Si=) yy if sec (12) 
j=1 
Step 5: Calculate the score for each alternative using formula (13). 
RI, = 2+ Ss: (13) 


Step 6: Rank the alternatives in descending order based on their scores. 
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PIV Method 
The PIV method employs six steps to rank alternatives (Mufazzal and Muzakkir, 2018). 
Step 1: Similar to Step 1 of the PS/ method. 
Step 2: Calculate the normalized values using formula (14). 
Xi 


m 2 
Dini Xi 


ti 
“ (14) 
Step 3: Calculate the normalized values considering the weights of the criteria according to formula (15). 


Vij = Wj X ny (15) 


Step 4: Evaluate the boundary weight index using formulas (16) and (17). 


U; = Vx vi if fj EB (16) 
U;=Vi-Vmin if fEC (17) 

Step 5: Determine the overall neighboring value range using formula (18). 
d,= iu, (18) 


Step 6: Rank the alternatives based on the principle that the best alternative is the one with the 
smallest deviation. 


Based on the implementation steps of the four methods PS/, SRP, RAM, and PIV as described 
above, the block diagram illustrating the application of these four methods for ranking options is presented 
in Figure 1. Notably, when applying the SRP, RAM, and PIV methods, an additional task of calculating 
weights for the criteria is required, whereas the PS/ method does not require the calculation of weights 
for the criteria. 


DECISION MATRIX 
Eq. (1), (2) Internally Eq. (9) 
rank 
the 
alternatives ! 
Eq. (3) ag Eq. (14) 
each 
criterion 
Eq. (4) 
Eq. (10) |= ae 
Eq. (5) | il ] 
Eq. (8) | Batt) Eq. (16), (17) 
Eq. (6) ! ! 
Eq. (12) Eq. (18) 
a ee 
Ea. (7) Eq. (13) 


RANK THE ALTERNATIVE 


PSI method SRP method RAMmethod PIV method Weight method 


Figure 1. Block diagram for ranking options 
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Results 


The selected course for this study is “Machine Manufacturing Technology Project’ because it pro- 
vides a flexible combination of direct classroom teaching and self-study at home. This allows students to 
directly interact with the fundamental knowledge conveyed by the lecturer, while also providing opportuni- 
ties for in-depth research, application, and experimentation with the acquired knowledge. The course is 
appealing as students can apply learned knowledge to practical situations, showcasing their creativity and 
problem-solving skills. Additionally, lecturer guidance during project execution is crucial, helping students 
gain a deeper understanding of the machine manufacturing process and enhancing teamwork skills. 
Furthermore, the ability to ask questions through applications is a significant advantage, enabling quick 
access to information and problem-solving. 

To view the grades of a course on the education system, each student needs to evaluate the lec- 
turer who taught them based on seventeen established criteria. Evaluation scores for each criterion range 
from 0 to 5, where a higher score indicates better fulfillment of that criterion. This means that all seventeen 
criteria fall into the B category. 


C1: The lecturer consistently prepares well for each class. 

This criterion emphasizes the importance of preparation for the teaching process. When lecturers 
invest time and effort in thorough preparation for classes, they can create a quality learning environment, 
enhancing students’ understanding and interest in the content. Preparation also helps lecturers be adapt- 
able to unexpected situations and improves effective communication. Students perceive this preparation 
through clear and accurate lectures, playing a vital role in building trust and respect for the lecturer. 


C2: Lessons are structured and organized systematically. 

The importance of structuring and organizing lessons lies in helping students understand and ab- 
sorb content in an organized and coherent manner. When lectures are systematically constructed, stu- 
dents can easily follow and organize information. Logical connections between teaching sections help 
students gain a deeper understanding and connect knowledge, forming a solid foundation. Lesson struc- 
ture also helps students predict lesson content, providing reassurance and confidence while promoting 
focus and learning efficiency. 


C3: The lecturer teaches with passion. 

Passion in teaching is not only a powerful motivator but also creates a positive learning environ- 
ment. When lecturers convey knowledge with enthusiasm, they inspire students and encourage creative 
thinking. The excitement of the lecturer can spread and stimulate curiosity and learning enthusiasm in 
students. Passion also helps lecturers overcome challenges in the teaching process, creating a positive 
environment that fosters students’ comprehensive development. 


C4: The lecturer motivates students to study this course. 

This highlights the importance of lecturers playing a significant role in motivating students. When 
lecturers stimulate students’ interest and desire to study, they create a positive environment, enhancing 
self-reliance and responsibility in the learning process. Motivation can arise from solving real-world prob- 
lems, applying knowledge to practical situations, or even the recognition of the value of the course for 
personal and career development. 


C5: Students are timely supported with the syllabus and official materials of the course. 

The importance of supporting students in accessing the syllabus and learning materials is crucial 
to ensure they have the necessary information. Timely support helps students overcome difficulties and 
enhances their understanding and application of knowledge. Providing official materials ensures consist- 
ency in information transmission, making it easier for students to follow and review. 


C6: Course materials are easily searchable and regularly updated. 

The ease of searching and updating course materials plays a significant role in optimizing students’ 
study time. When materials are well-organized and easy to search, students can quickly access the infor- 
mation they need, enhancing efficiency in the review and grasp of knowledge. Regular updates ensure 
that materials reflect the latest progress in the course field, helping students maintain continuity and apply 
knowledge to real-life situations. 
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C7: Lecturers provide sufficient, clear overall content information and distribute the course sched- 
ule before teaching. 

Providing sufficient and clear information before class emphasizes the importance of preparation 
and transparency in the teaching process. Students can better prepare when they know the content and 
schedule of the course, allowing them to manage their study time more effectively. This transparency 
creates a collaborative learning environment, where lecturers and students can work together effectively. 


C8: Lecturers provide sufficient, clear course outcomes before teaching. 

Course outcomes are an essential indicator of the knowledge and skills students will acquire af- 
ter completing the course. Clear communication about course outcomes helps students understand the 
course’s objectives and shape an appropriate study plan. Simultaneously, lecturers can align expecta- 
tions and evaluate learning outcomes fairly and transparently. 


C9: Lecturers start class on time, ensuring the time and teaching volume follow the course syllabus. 

Ensuring timeliness and the teaching volume not only creates a disciplined learning environment 
but also helps students manage their study time effectively. When lecturers start classes on time and ad- 
here to the course syllabus, students can rely on the schedule to prepare and actively participate in class- 
es. Accuracy and consistency in teaching contribute to the stability and quality of the learning process. 


C10: Lecturers guide you on study methods when starting the course. 

The importance of guiding students on study methods at the beginning of a course is to help them 
create a study plan that suits their learning style. This not only enhances students’ autonomy in the learn- 
ing process but also lays the foundation for developing self-learning skills, a crucial factor in their overall 
learning journey. 


C11: The lecturer’s teaching method is understandable and effective. 

The effectiveness and clarity of teaching methods are decisive factors in students’ understanding 
and absorption. When lecturers apply teaching methods appropriate to the target audience, information 
delivery becomes smooth and vibrant. This method helps students form a comprehensive view of the 
course and enhances the ability to apply knowledge to real-life situations, creating a tight connection 
between theory and practice. 


C12: Lecturers create opportunities for you to actively participate in the learning process. 

The importance of lecturers creating opportunities for students to actively participate lies in encour- 
aging interaction and positivity in the learning process. When students have the opportunity to discuss, 
ask questions, and contribute opinions, they not only enhance understanding but also develop commu- 
nication and critical thinking skills. Active participation helps students become confident and enjoy the 
learning process, creating a motivating and supportive learning environment. 


C13: Student academic performance is evaluated through various forms and aligns with the course 
outcomes. 

The diversity in evaluating academic performance and alignment with course outcomes is crucial 
to ensuring accuracy and fairness in the evaluation process. When lecturers use various assessment 
methods such as exams, projects, presentations, or group discussions, students have the opportunity to 
demonstrate their skills and knowledge from different perspectives. This not only strengthens the learning 
process but also creates conditions for the multidimensional development of subject competencies. 


C14: Feedback from tests and evaluations helps you improve academic performance. 

Feedback is an essential part of the learning process, helping students understand more about their 
level of success and areas for improvement. When lecturers provide detailed and constructive feedback, 
students can self-assess their abilities and performance, thereby seeking ways to improve. Accurate and 
constructive feedback not only supports the current learning process but also serves as a foundation for 
continuous development and lifelong learning. 


C15: The lecturer’s teaching method stimulates students’ critical thinking and problem-solving abilities. 
The importance of stimulating students’ critical thinking and problem-solving abilities lies in helping 
them develop crucial skills for the real world. When lecturers create lectures and activities that encour- 
age logical thinking, information analysis, and problem-solving, students not only learn a subject but also 
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develop essential skills for their careers and daily lives. 


C16: | learned a lot about the course from the lecturer. 
This statement reflects the success of conveying knowledge and creating a positive learning expe- 
rience. When students perceive the learning and personal development through the lecturer, they tend to 
positively evaluate the learning process and consider the course as an essential part of their educational 
journey. 


C17: General Evaluation - The lecturer is highly suitable for teaching this course. 


The overall evaluation is the final and comprehensive result of the lecturer’s success in teaching 
this course. 
Data extraction from the university's training management system on the scores of the seventeen 
criteria for eight lecturers teaching the Machine Manufacturing Technology Project course is presented in 
Table 2. 


Table 2. Synthesized comments from students for each lecturer (source: author’s compilation) 


Lecturers (C1 


L1 
L2 
L3 
L4 
L5 
L6 
L7 
L8 


4.67 
4.58 
4.66 
4.51 
4.82 
5 
4.76 
4.67 


C2 
47 
4.54 
4.63 
4.49 
477 
45 
4.68 
4.63 


C3 


4.76 
4.63 


C4 
47 
4.46 
4.63 
4.49 
4.85 
45 
4.68 
4.59 


C5 


4.72 
4.52 


4.76 
4.52 


45 
4.8 
4.53 


C8 


4.72 
4.53 


45 
48 
4.55 


4.76 
4.56 


C11 
4.67 
4.55 


45 
4.76 
4.64 


45 
48 
4.66 


C13 


4.72 
4.66 


C14 


4.5 
4.64 
4.66 


C15 


45 
4.76 
4.66 


4.72 
4.73 


4.76 
4.73 


Currently, the university's management level relies on the data of the seventeen criteria in Table 2 
to calculate the average scores for each lecturer and then rank the lecturers in Table 3. This result will be 
used for comparison with the lecturer rankings using the MCDM methods that this research is conducting. 


Table 3. Lecturer rankings using the traditional method 


Lecturers Score Rank 
L1 4.69 5 
L2 4.55 f 
L3 4.72 4 
L4 4.54 8 
L5 4.94 | 
L6 4.76 2 
Ly 4.74 3 
L8 4.62 6 
Applying the PSI method 


Normalized values have been calculated using formulas (1) and (2), resulting in Table 4. 


Table 4. Normalized values in the PSI method 


Lecturers 


1 
2 
3 
4 
L5 
6 
7 
8 


C1 
0.9340 
0.9160 
0.9320 
0.9020 
0.9640 
1.0000 
0.9520 
0.9340 


C2 
0.9853 
0.9518 
0.9706 
0.9413 
1.0000 
0.9434 
0.9811 
0.9706 


C3 
0.9400 
0.9080 
0.9360 
0.9020 
0.9760 
1.0000 
0.9520 
0.9260 


C4 
0.9691 
0.9196 
0.9546 
0.9258 
1.0000 
0.9278 
0.9649 
0.9464 


C5 
0.9400 
0.9020 
0.9520 
0.9080 
0.9720 
1.0000 
0.9440 
0.9040 


3) 
0.9460 
0.9100 
0.9420 
0.9040 
0.9840 
1.0000 
0.9520 
0.9040 


C7 
0.9340 
0.9200 
0.9360 
0.9040 
1.0000 
0.9000 
0.9600 
0.9060 


C8 
0.9400 
0.9080 
0.9480 
0.9080 
0.9700 
1.0000 
0.9440 
0.9060 


cg 
0.9460 
0.9140 
0.9520 
0.9100 
1.0000 
0.9000 
0.9600 
0.9100 


C10 
0.9460 
0.9040 
0.9360 
0.9040 
1.0000 
1.0000 
0.9520 
0.9120 


C11 
0.9340 
0.9100 
0.9480 
0.9160 
1.0000 
0.9000 
0.9520 
0.9280 


C12 
0.9340 
0.9200 
0.9480 
0.9140 
1.0000 
0.9000 
0.9600 
0.9320 


C13 
0.9400 
0.9100 
0.9480 
0.9100 
1.0000 
1.0000 
0.9440 
0.9320 


C14 
0.9400 
0.8960 
0.9480 
0.9140 
1.0000 
0.9000 
0.9280 
0.9320 


C15 
0.9400 
0.9140 
0.9520 
0.9100 
1.0000 
0.9000 
0.9520 
0.9320 


C16 
0.9340 
0.9200 
0.9520 
0.9160 
0.9960 
1.0000 
0.9440 
0.9460 


C17 
0.9340 
0.9300 
0.9520 
0.9140 
0.9940 
1.0000 
0.9520 
0.9460 
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Applying formula (3) to calculate the average values of standardized data (n); priority values from 
the average values (.) calculated using formula (4); deviation in priority values (<j) calculated using 
formula (5); overall priority values for criteria calculated using formula (6). All calculated values have been 
summarized in Table 5. 


Table 5. Some parameters in PS! 
“Pa, Xl C2 C3) Chl! OC8llCOUCOsC“(<éi‘i :SC<i<iSCCi‘iSCi2siHRC(iC(‘iéiSCCASCCA SC 
N 0.9418 0.9680 0.9425 0.9510 0.9403 0.9428 0.9325 0.9405 0.9365 0.9443 0.9360 0.9385 0.9480 0.9323 0.9375 0.9510 0.9528 
, 0.0065 0.0031 0.0077 0.0051 0.0085 0.0092 0.0081 0.0078 0.0082 0.0106 0.0069 0.0068 0.0086 0.0076 0.0071 0.0070 0.0064 


2} 0.9935 0.9969 0.9923 0.9949 0.9915 0.9908 0.9919 0.9922 0.9918 0.9894 0.9931 0.9932 0.9914 0.9924 0.9929 0.9930 0.9936 
B, 0.0589 0.0591 0.0588 0.0590 0.0588 0.0587 0.0588 0.0588 0.0588 0.0586 0.0588 0.0589 0.0587 0.0588 0.0588 0.0588 0.0589 


The preference selection index (q,) for each lecturer has been calculated using formula (7), result- 


ing in Table 6. The ranking of each lecturer has also been arranged based on their g, values and has been 
placed in the last column of this table. 


Table 6. Preference selection index (q,) and rankings of lecturers 


Lecturers 0i Rank 
L1 0.9433 5 
L2 0.9149 7 
L3 0.9475 4 
L4 0.9120 8 
L5 0.9915 1 
L6 0.9571 2 
L7 0.9526 3 
L8 0.9275 6 
Applying the SRP method 


The internal ranking of lecturers has been carried out according to step 2, resulting in Table 7. 
Table 7. Internal rankings of lecturers using the SRP method 


Lecturers C1 C2 C3 C4 C5 C6 C7 C8 co 6C10) «6Cl)«=6C12—s C138 C14. C15 C16 Ss C17 


L1 4 2 4 2 5 4 4 5 4 3 4 4 4 3 3 6 5 
L2 6 5 7 8 8 6 5 6 5 6 7 6 6 8 5 7 6 
L3 5 4 5 4 3 5 3 3 3 4 3 3 2 2 2 3 3 
L4 7 7 8 7 6 7 7 6 6 6 6 7 6 6 6 8 7 
L5 2 1 2 1 2 2 1 2 1 1 1 1 1 1 1 2 2 
L6 1 6 1 6 1 1 8 1 7 1 8 8 1 7 7 1 1 
L7 3 3 3 3 4 3 2 4 2 2 2 2 3 5 2 5 3 
L8 4 4 6 5 7 7 6 7 6 5 5 5 5 4 4 4 4 


Individual scores (S_) for each lecturer have been calculated using (8), where the weights of the criteria 


are chosen to be equal, meaning each criterion has a weight of 1/17. The scores and rankings of lecturers 
have been consolidated in Table 8. 
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Lecturers 


Table 8. Scores and rankings of lecturers 


Lecturers 


L1 
L2 
L3 
L4 
L5 
L6 
L7 
L8 


Applying the RAM method 
Normalized values have been calculated using (9), resulting in Table 9. 
Table 9. Normalized values in the RAM method 


C1 
0.1240 
0.1216 
0.1237 
0.1197 
0.1280 
0.1327 
0.1264 
0.1240 


C2 
0.1272 
0.1229 
0.1253 
0.1215 
0.1291 
0.1218 
0.1267 
0.1253 


C3 
0.1247 
0.1204 
0.1241 
0.1196 
0.1294 
0.1326 
0.1263 
0.1228 


C4 
0.1274 
0.1209 
0.1255 
0.1217 
0.1314 
0.1220 
0.1268 
0.1244 


C5 
0.1250 
0.1199 
0.1266 
0.1207 
0.1292 
0.1329 
0.1255 
0.1202 


C6 
0.1254 
0.1207 
0.1249 
0.1199 
0.1305 
0.1326 
0.1262 
0.1199 


C7 
0.1252 
0.1233 
0.1255 
0.1212 
0.1340 
0.1206 
0.1287 
0.1214 


S 


3.8824 
6.2941 
3.3529 
6.6471 
1.4118 
3.8824 
3.0000 
5.1765 


C8 
0.1249 
0.1207 
0.1260 
0.1207 
0.1289 
0.1329 
0.1255 
0.1204 


C9 
0.1263 
0.1220 
0.1271 
0.1215 
0.1335 
0.1201 
0.1281 
0.1215 


C10 
0.1252 
0.1197 
0.1239 
0.1197 
0.1324 
0.1324 
0.1260 
0.1207 


Rank 


C11 
0.1247 
0.1215 
0.1266 
0.1223 
0.1335 
0.1202 
0.1271 
0.1239 
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C12 
0.1244 
0.1225 
0.1263 
0.1217 
0.1332 
0.1199 
0.1279 
0.1241 


C13 
0.1239 
0.1200 
0.1250 
0.1200 
0.1319 
0.1319 
0.1245 
0.1229 


C14 
0.1260 
0.1201 
0.1271 
0.1226 
0.1341 
0.1207 
0.1244 
0.1250 


C15 
0.1253 
0.1219 
0.1269 
0.1213 
0.1333 
0.1200 
0.1269 
0.1243 


C16 
0.1228 
0.1209 
0.1251 
0.1204 
0.1309 
0.1314 
0.1241 
0.1243 


C17 
0.1225 
0.1220 
0.1249 
0.1199 
0.1304 
0.1312 
0.1249 
0.1241 


Normalized values, considering the weights of criteria, have been calculated using (10). Here, the 
weights of each criterion have also been chosen to be equal (1/17), resulting in Table 10. 


Table 10. Normalized values considering the weights of criteria (in the RAM method) 


Lecturers 
L1 
L2 
L3 
L4 
L5 
L6 
L7 
L8 


C1 
0.0073 
0.0072 
0.0073 
0.0070 
0.0075 
0.0078 
0.0074 
0.0073 


C2 
0.0075 
0.0072 
0.0074 
0.0071 
0.0076 
0.0072 
0.0075 
0.0074 


C3 
0.0073 
0.0071 
0.0073 
0.0070 
0.0076 
0.0078 
0.0074 
0.0072 


C4 
0.0075 
0.0071 
0.0074 
0.0072 
0.0077 
0.0072 
0.0075 
0.0073 


C5 
0.0074 
0.0071 
0.0074 
0.0071 
0.0076 
0.0078 
0.0074 
0.0071 


C6 
0.0074 
0.0071 
0.0073 
0.0071 
0.0077 
0.0078 
0.0074 
0.0071 


C7 
0.0074 
0.0073 
0.0074 
0.0071 
0.0079 
0.0071 
0.0076 
0.0071 


C8 
0.0073 
0.0071 
0.0074 
0.0071 
0.0076 
0.0078 
0.0074 
0.0071 


cg 
0.0074 
0.0072 
0.0075 
0.0071 
0.0079 
0.0071 
0.0075 
0.0071 


C10 
0.0074 
0.0070 
0.0073 
0.0070 
0.0078 
0.0078 
0.0074 
0.0071 


C11 
0.0073 
0.0071 
0.0074 
0.0072 
0.0079 
0.0071 
0.0075 
0.0073 


C12 
0.0073 
0.0072 
0.0074 
0.0072 
0.0078 
0.0071 
0.0075 
0.0073 


C13 
0.0073 
0.0071 
0.0074 
0.0071 
0.0078 
0.0078 
0.0073 
0.0072 


C14 
0.0074 
0.0071 
0.0075 
0.0072 
0.0079 
0.0071 
0.0073 
0.0074 


C15 
0.0074 
0.0072 
0.0075 
0.0071 
0.0078 
0.0071 
0.0075 
0.0073 


C16 
0.0072 
0.0071 
0.0074 
0.0071 
0.0077 
0.0077 
0.0073 
0.0073 


C17 
0.0072 
0.0072 
0.0073 
0.0071 
0.0077 
0.0077 
0.0073 
0.0073 


Total normalized scores, considering the weights of criteria, have been calculated using (11) and (12); 
Individual scores for each lecturer have been obtained according to (13). Table 11 synthesizes the calcu- 
lated values and the rankings of lecturers based on their scores (RI). 
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Table 11. Some parameters in the RAM method and rankings of lecturers 


Lecturers oe S, RI, Rank 
L1 0.125002 1.4577 5 
L2 0.121237 1.4564 7 
L3 0.125559 1.4579 4 
L4 0.120848 0 1.4563 8 
L5 0.131401 1.4599 1 
L6 0.126821 1.4584 2 
L7 0.126235 1.4582 3 
L8 0.122897 1.4570 6 
Applying the PIV method 


Lecturers 
L1 
L2 
L3 
L4 
L5 
L6 
L7 
L8 


Standardized values have been calculated using (14), consolidated in Table 12. 
Table 12. Standardized values in the PIV method 


C1 C2 
0.3505 0.3598 
0.3437 0.3475 
0.3497 0.3544 
0.3385 0.3437 
0.3617 0.3652 
0.3753 0.3445 
0.3572 0.3583 
0.3505 0.3544 


C3 
0.3524 
0.3404 
0.3509 
0.3382 
0.3659 
0.3749 
0.3569 
0.3472 


C4 
0.3601 
0.3417 
0.3548 
0.3440 
0.3716 
0.3448 
0.3586 
0.3517 


C5 
0.3532 
0.3390 
0.3578 
0.3412 
0.3653 
0.3758 
0.3547 
0.3397 


C6 
0.3545 
0.3411 
0.3530 
0.3388 
0.3688 
0.3748 
0.3568 
0.3388 


C7 
0.3539 
0.3486 
0.3547 
0.3425 
0.3789 
0.3410 
0.3638 
0.3433 


C8 
0.3532 
0.3411 
0.3562 
0.3411 
0.3644 
0.3757 
0.3547 
0.3404 


ie) 
0.3569 
0.3449 
0.3592 
0.3433 
0.3773 
0.3396 
0.3622 
0.3433 


c1i0.— C11 
0.3539 0.3526 
0.3382 0.3436 
0.3502 0.3579 
0.3382 0.3458 
0.3741 0.3775 
0.3741 0.3398 
0.3562 0.3594 
0.3412 0.3504 


C12 
0.3517 
0.3464 
0.3570 
0.3442 
0.3765 
0.3389 
0.3615 
0.3509 


C13 
0.3504 
0.3392 
0.3533 
0.3392 
0.3727 
0.3727 
0.3519 
0.3474 


C14 
0.3563 
0.3396 
0.3593 
0.3464 
0.3790 
0.3411 
0.3517 
0.3533 


C15 
0.3543 
0.3445 
0.3588 
0.3430 
0.3769 
0.3392 
0.3588 
0.3513 


C16 
0.3471 
0.3419 
0.3538 
0.3404 
0.3701 
0.3716 
0.3508 
0.3515 


C17 
0.3464 
0.3450 
0.3531 
0.3390 
0.3687 
0.3709 
0.3531 
0.3509 


Lecturers 
L1 
L2 
L3 
L4 
L5 
L6 
L7 
L8 


Lecturers 
L1 
L2 
L3 
L4 
L5 
L6 
L7 
L8 


Standardized values, considering the weights of criteria, have been calculated using (15), resulting 
in Table 13. Here, the weights of each criterion have also been chosen to be equal (1/17). 


Table 13. Standardized values considering the weights of criteria (in the PIV method) 


C1 C2 
0.0206 0.0212 
0.0202 0.0204 
0.0206 0.0208 
0.0199 0.0202 
0.0213 0.0215 
0.0221 0.0203 
0.0210 0.0211 
0.0206 0.0208 


C3 
0.0207 
0.0200 
0.0206 
0.0199 
0.0215 
0.0221 
0.0210 
0.0204 


C4 
0.0212 
0.0201 
0.0209 
0.0202 


C5 
0.0208 
0.0199 
0.0210 
0.0201 


C6 
0.0209 
0.0201 
0.0208 
0.0199 


0.0219 0.0215 0.0217 


0.0203 
0.0211 
0.0207 


0.0221 
0.0209 
0.0200 


0.0220 
0.0210 
0.0199 


C7 
0.0208 
0.0205 
0.0209 
0.0201 
0.0223 
0.0201 
0.0214 
0.0202 


C8 
0.0208 
0.0201 
0.0210 
0.0201 
0.0214 
0.0221 
0.0209 
0.0200 


C9 
0.0210 
0.0203 
0.0211 
0.0202 
0.0222 
0.0200 
0.0213 
0.0202 


C1i0.— C11 
0.0208 0.0207 
0.0199 0.0202 
0.0206 0.0211 
0.0199 0.0203 
0.0220 0.0222 
0.0220 0.0200 
0.0210 0.0211 
0.0201 0.0206 


C12 
0.0207 
0.0204 
0.0210 
0.0202 
0.0221 
0.0199 
0.0213 
0.0206 


C13 
0.0206 
0.0200 
0.0208 
0.0200 
0.0219 
0.0219 
0.0207 
0.0204 


C14 
0.0210 
0.0200 
0.0211 
0.0204 
0.0223 
0.0201 
0.0207 
0.0208 


C15 
0.0208 
0.0203 
0.0211 
0.0202 
0.0222 
0.0200 
0.0211 
0.0207 


C16 
0.0204 
0.0201 
0.0208 
0.0200 
0.0218 
0.0219 
0.0206 
0.0207 


The weight proximity index has been calculated using (16) and (17), resulting in Table 14. 
Table 14. Weight proximity index values in the PIV method 


C1 C2 
0.0015 0.0003 
0.0019 0.0010 
0.0015 0.0006 
0.0022 0.0013 
0.0008 0.0000 
0.0000 0.0012 
0.0011 0.0004 
0.0015 0.0006 


C3 
0.0013 
0.0020 
0.0014 
0.0022 
0.0005 
0.0000 
0.0011 
0.0016 


C4 
0.0007 
0.0018 
0.0010 
0.0016 
0.0000 
0.0016 
0.0008 
0.0012 


C5 
0.0013 
0.0022 
0.0011 
0.0020 
0.0006 
0.0000 
0.0012 
0.0021 


C6 
0.0012 
0.0020 
0.0013 
0.0021 
0.0004 
0.0000 
0.0011 
0.0021 


C7 
0.0015 
0.0018 
0.0014 
0.0021 
0.0000 
0.0022 
0.0009 
0.0021 


C8 
0.0013 
0.0020 
0.0011 
0.0020 
0.0007 
0.0000 
0.0012 
0.0021 


cg 
0.0012 
0.0019 
0.0011 
0.0020 
0.0000 
0.0022 
0.0009 
0.0020 


C10. C11 
0.0012 0.0015 
0.0021 0.0020 
0.0014 0.0012 
0.0021 0.0019 
0.0000 0.0000 
0.0000 0.0022 
0.0011 0.0011 
0.0019 0.0016 
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C12 
0.0015 
0.0018 
0.0012 
0.0019 
0.0000 
0.0022 
0.0009 
0.0015 


C13 
0.0013 
0.0020 
0.0011 
0.0020 
0.0000 
0.0000 
0.0012 
0.0015 


C14 
0.0013 
0.0023 
0.0012 
0.0019 
0.0000 
0.0022 
0.0016 
0.0015 


C15 
0.0013 
0.0019 
0.0011 
0.0020 
0.0000 
0.0022 
0.0011 
0.0015 


C16 
0.0014 
0.0017 
0.0010 
0.0018 
0.0001 
0.0000 
0.0012 
0.0012 


C17 
0.0204 
0.0203 
0.0208 
0.0199 
0.0217 
0.0218 
0.0208 
0.0206 


C17 
0.0014 
0.0015 
0.0010 
0.0019 
0.0001 
0.0000 
0.0010 
0.0012 


345 


Trung D. D. et al. (2024). Building a Ranking System for Lecturers Based on Student Evaluations in Teaching a Specific 
Course: A Case Study at a University in Vietnam, International Journal of Cognitive Research in Science, Engineering and 
Education (ISCRSEE), 12(2), 335-350. 


The overall nearness value (d') for each lecturer has been calculated according to (18), resulting in 
Table 15. The ranking of each lecturer has also been determined based on their d. values and has been 
consolidated in the last column of this table. 


Table 15. The d; values and rankings of each lecturer 


Lecturers d. rank 
L1 0.0213 5 
L2 0.0319 r( 
L3 0.0197 4 
L4 0.0330 8 
L5 0.0032 1 
L6 0.0161 2 
L7 0.0178 3 
L8 0.0272 6 


Therefore, the ranking of lecturers using the PS/, SRP RAM, and PIV methods has been complet- 
ed. The results of the rankings using the traditional method that the university is currently using are also 
used for comparison with the rankings obtained using these four methods. Figure 2 shows the lecturer 
rankings using different methods when the weights of the criteria are equal. 


10 
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Figure 2. Rankings of lecturers using different methods when the weights of the criteria are equal 


We observe that the rankings of lecturers when assessed by the PS/, RAM, and P/V methods are 
entirely identical and also completely match the rankings obtained through the traditional method used at 
the university. There is a slight difference in the rankings of lecturers when assessed by the SRP method 
compared to when using other methods. However, the lecturer rated as the best and the one rated as 
the worst when ranked by the SRP method always aligns with the rankings obtained using other ranking 
methods. The difference in rankings when using the SRP method compared to using other methods is 
explained by the fact that SRP only uses natural numbers for internal ranking of lecturers, reducing adapt- 
ability compared to normalizing data to generate real numbers (both natural and decimal) when using 
other methods. 

The results achieved and the observations made apply when all selected criteria have equal 
weights (all equal to 1/17). However, to answer the question about the accuracy of results when weights 
of criteria are calculated using different methods, ranking of lecturers is necessary. Two methods, Entropy 
and MEREC, have been proposed with high accuracy in calculating weights for criteria (Trung and Thinh, 
2021). They have also been widely used in recent studies (Hoang, 2023; Trung, 2021a; Trung, 2021b; Le 
et al., 2022), and they were selected for application in this research. The weights of each of the seventeen 
criteria for each lecturer, as determined by the Entropy and MEREC methods, can be found in Table 16. 
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Table 16. Weights of criteria 


Weight 
method 


Entropy 0.0588 0.0591 0.0588 0.0591 0.0588 0.0588 0.0589 0.0588 0.0589 0.0587 0.0589 0.0588 0.0587 0.0589 0.0588 0.0587 0.0586 
MEREC 0.0648 0.0422 0.0657 0.0503 0.0619 0.0624 0.0527 0.0556 0.0593 0.0645 0.0587 0.0628 0.0608 0.0592 0.0611 0.0559 0.0622 


C1 C2 C3 C4 C5 C6 C7 C8 co. 6C10) «= C11 C12, 0138 C14 C15 C16 SC 


Ranking lecturers using the SRP. RAM, and PIV methods when the weights of criteria are calculated 
using the Entropy and MEREC methods has also been conducted similarly to when the weights of all cri- 
teria are equal, as we have previously performed. Figures 3 and 4 respectively represent the rankings of 
lecturers when the weights of criteria are calculated using the Entropy and MEREC methods. It is impor- 
tant to reiterate that since the PS/ method is not related to the weights of criteria, the rankings of lecturers 
using this method are independent of the weights of criteria. 
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Figure 3. Rankings of lecturers using different methods when the weights of criteria are calculated using the 
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Figure 4. Rankings of lecturers using different methods when the weights of criteria are calculated using the 
MEREC method 


From the observations in Figures 3 and 4, we also notice that the rankings of lecturers are entirely 
identical when ranked by the PS/, RAM, PIV methods, and the traditional method of the university, regard- 
less of the weights of criteria. Furthermore, the ranking of each lecturer remains unchanged when the 
weights of criteria change (review all three Figures 2, 3, and 4). Moreover, the complete dissimilarity in 
rankings of lecturers when ranked by the SRP method compared to using other methods persists, as ob- 
served in Figure 2 above. However, the lecturer rated as Rank 1 and the lecturer rated as Rank 8 remain 
entirely identical when using different methods. All of these findings indicate that the application of the four 
methods PSI, SRP. RAM, and PIV is entirely appropriate for identifying the best lecturer for a course. The 
lecturer deemed least suitable for teaching a course will also be identified when applying these methods. 
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Discussions 


The performance evaluation of university lecturers is a crucial issue in today’s higher education 
system. Our study has demonstrated that methods such as PS/, SRP. RAM, and PIV, alongside traditional 
ranking methods, lead to comparable assessment outcomes for lecturers teaching mechanical engineer- 
ing design projects. This consistency not only affirms the effectiveness of these methods but also under- 
scores the reliability of the evaluation process. 

We have identified that lecturers ranked as most and least suitable in teaching this course main- 
tain their positions when different evaluation methods are applied throughout the academic year. This 
indicates the consistency and stability of the methods in assessing lecturer performance. The ability to 
expand the application of these evaluation methods from mechanical engineering design projects to other 
teaching fields not only enhances flexibility but also ensures the feasibility and effectiveness of the evalu- 
ation process. 

In summary, our research not only addresses a specific issue but also opens avenues to explore 
and apply more effective teacher evaluation methods across different teaching domains, contributing to 
continuous improvement in the quality of higher education. 


Conclusions 


v¥ The PSI, RAM, PIV methods, and the traditional ranking approach all lead to similar lecturer rank- 
ings, demonstrating consistency in evaluating their performance. 

v The lecturer identified as the most suitable and least suitable for teaching the machine manu- 
facturing technology project consistently maintains this position when using all four evaluation 
methods, including PS/, SRP RAM, PIV, and the traditional ranking approach. 

v There is potential to extend the application of the evaluation methods used in the machine manu- 
facturing technology project to rank lecturers when teaching other courses, providing flexibility in 
the evaluation process. 

v When determining the importance of different criteria, such as according to the opinions of uni- 
versity administrators, various methods for calculating weights, such as PIPRECIA (Dragisa et 
al., 2021), LOPCOW (Ecer and Pamucar, 2022), can be employed to ensure accuracy and adapt- 
ability to specific environments. 

v Insummary, the research has provided a comprehensive insight into the lecturer evaluation pro- 
cess in the machine manufacturing technology project and opens up opportunities to extend the 
application of these methods to other teaching areas. 
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